Papers with multimodal memory retrieval
Enabling Chatbots with Eyes and Ears: An Immersive Multimodal Conversation System for Dynamic Interactions (2025.acl-long)
Copied to clipboard
| Challenge: | Multimodality has been explored in multi-party and multi-session conversations, but task-specific constraints have hindered its seamless integration into dynamic, natural conversations. |
| Approach: | They propose a multimodal conversation dataset and a model with multimodal memory retrieval to equip chatbots with "eyes and ears" they aim to integrate multimodality into chatbot interactions by integrating visual and auditory inputs into the chatbot. |
| Outcome: | The proposed model demonstrates the ability to engage in long-term conversations with multiple speakers in complex, real-world-like settings, effectively processing visual and auditory inputs to understand and respond appropriately. |